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Abstract 

We consider mutual information between release times and capture times for a set of M identical 
quanta traveling independently from a source to a target. The quanta are immediately captured upon 
arrival, first-passage times are assumed independent and identically distributed and the quantum 
emission times are constrained by a deadline. The primary application area is intended to be 
inter /intracellular molecular signaling in biological systems whereby an organelle, cell or group of 
cells must deliver some message (such as transcription or developmental instructions) over distance 
with reasonable certainty to another organelles, cells or group of cells. However, the model can 
also be applied to communications systems wherein indistinguishable signals have random transit 
latencies. 

1 Introduction 

Biological systems are networks of intercommunicating elements at whatever level one cares 
to consider - (macro)molecules, cells, tissues, organisms, populations, microbiomes, ecosys- 
tems, and so on. It is no wonder therefore that communication theorists have plied their 
trade heavily in this scientific domain (for a recent review, see [l]). Biological systems offer 
a dizzying array of processes and phenomena through which the same and different tasks, 
communication or otherwise, might be accomplished (see, for example, f2|-l7|). Identifying 
the underlying mechanisms (signaling modality, signaling agent, signal transport, and so on) 
as well as the molecules and structures implementing the mechanisms is no small undertak- 
ing. Consequently, experimental biologists use a combination of prior knowledge and what 
can only be called instinct to choose those systems on which to expend effort. Guidance may 
be sought from evolutionary developmental biology - a field that compares the developmen- 
tal processes of different organisms to determine their ancestral relationship and to discover 
how developmental processes evolved. Insights may be gained by using statistical machine 
learning techniques to analyze heterogeneous data such as the biomedical literature and 
the output of so-called "omics" technologies - genomics (genes, regulatory, and non-coding 
sequences), transcriptomics (RNA and gene expression), proteomics (protein expression), 
metabolomics (metabolites and metabolic networks), pharmacogenomics (how genetics af- 
fects hosts' responses to drugs), and physiomics (physiological dynamics and functions of 
whole organisms). 

Typically, the application of communication theory to biology starts by selecting a can- 
didate system whose components and operations have been already elucidated to varying 
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degrees using methods in the experimental and/or computational biology toolbox [8|[9] and 
then applying communication theoretic methods [H[7|, [l0}jl2] . However, we believe that com- 
munication theory in general and information theory in particular are not merely system 
analysis tools for biology. That is, given energy constraints and some general physics of the 
problem, an information-theoretic treatment can be used to provide outer bounds on infor- 
mation transfer in a mechanism-blind manner. Thus, rather than simply elucidating and 
quantifying known biology, communication theory can winnow the plethora of possibilities 
(or even suggest new ones) amenable to experimental and computational pursuit. Likewise, 
general application of communication-theoretic principles to biology affords a new set of ap- 
plication areas for communication theorists. Some aspects of the potential for communication 



theory as a new lens on biological systems are explored in 13 . 

In this light, here we devise an abstraction that encompasses a myriad of biological 
processes and phenomena, utilize it to devise a simpler model suitable for communication- 
theoretic investigations, and analyze the resultant model using ideas discussed in seemingly 



unrelated work, namely the capacity of timing channels 14 . Numerous scenarios in bi- 
ology that involve the transmission of information can be synthesized and summarized as 
inscribed matter is sent by an emitter, moves through a medium, and arrives eventually at 
its destination receptor where it is interpreted. 

Scenarios illustrating the complexity and diversity that our abstraction attempts to cap- 
ture include the following: 

• messenger RNA molecules (mRNAs) that are transcribed from the genome migrate 
from the nucleus to the cytoplasm where they are translated by the ribosome into 
proteins. 

• Molecules of the neurotransmitter acetylcholine (Ach) that are released by the presy- 
naptic neuron terminal diffuse through the synaptic cleft and bind to nicotinic Ach 
receptors on the motor end plate. 

• Ions, molecules, organelles, bacteria and viruses that are present in one cell are shipped 
through a thin membrane channel (tunneling nanotube) to the connected cell where 
they elicit a physiological response. 

• Membrane-bound vesicles that contain a variety of materials and substances translocate 
through the cytoplasm to the cell membrane where release their contents into the 
extracellular environment. 

• Malignant cells that have escaped the confines of a tissue circulate through the blood- 
stream to other sites where they re-penetrate the vessel walls and can seed a new 
tumor. 

• Chemicals factors that are secreted or excreted by an individual travel outside the 
body where they are sensed by a member of the same species triggering a social or 
behavioral response. 

Although the abstraction accommodates a wide range of spatiotemporal scale and types 
of emitters, inscribed matter, and receptors, it neglects many biologically important fea- 
tures. For example, the suite of signaling quanta - molecules, macromolecular complexes, 
organelles, cells, and so on - that are released is not necessarily the same as that which 



reaches the target because some may be changed (eukaryotic mRNAs are modified post- 
transcriptionally), some may be removed (Ach can be degraded by the enzyme Ach-esterase), 
some never arrives (the random path produced by diffusion may result in a trajectory that 
leads away from the target [l5]), and so on. The movement of inscribed matter may be 
passive or active, may or may not require energy and so on. 

Despite its limitations, the abstraction does embody a number of salient features. Typi- 
cally, information is thought to be conveyed via numbers of signaling quanta (concentration). 
Thus, what amount to dose-response curves are the norm for a variety of experimental bi- 
ology studies [7] and clever theoretical workups (e.g., [I6]). However, as was shown in an 
entirely different domain and unrelated work [m] , timing of emissions could in principle also 
convey information. Clearly, this possibility cannot be ignored if our aim is to attempt to 
provide bounds on what "a cell can tell the world." Under certain conditions, perhaps timing 
is a useful complement to concentration or even essential. Alternatively, timing might some- 
times be energetically unfavorable and its use unlikely. In either case, information-theoretic 
bounds would help guide biological inquiry. 

Our emitter- receptor system is also, at least in part, motivated by fundamental "sys- 
tems" problems in biology such as development, wherein undifferentiated cells are "told" 
what to become by a combination of internal programming and extracellular milieu signals 
- and in turn tell other cells what to become [TT] . Thus, communication within and between 
cells plays a vital role in the development (embryogenesis) , maintenance (tissue homeosta- 
sis), subversion (disorders such as cancer, infiammation, infections) and decline (aging) of 
multicellular forms and systems. 

Unfortunately, the detailed physics of even this seemingly simple abstraction are fraught 
with a variety of complications. As indicated above, free-space diffusive first passage times 
are generally not at all well-behaved. There may be deletions (a quantum is captured and 
destroyed by "lysing" agents) or the first passage density may be heavy-tailed to the point 



that sometimes some of the inscribed matter may never arrive at the receptor site 15 . Here 
we will ignore both complications. Random deletions can only reduce information transfer, 
so assuming quanta survive transit provides an upper bound. Likewise for heavy-tailed 
first passage densities, cells emitting signaling quanta into constrained extracellular (or even 
more tightly constrained intracellular) media, arrival with finite mean first passage time 
seems reasonable. 

However, the most technically difficult complication - and one which cannot be ignored - 
is quanta indistinguishability. Which emission corresponds to which arrival can be ambigu- 
ous. That is, if emissions occur at times {Tj} and the corresponding arrivals occur at {Si}, 
then all the receiver has available is {St}, the time-ordered version of the arrivals. Thus, our 
major task is to derive expressions for /(S; T) and thence maxj^ J(S; T). 

In what follows we first formally define the problem, provide some simplifying symmetry 
assumptions, explore their implications and then derive expressions for the mutual informa- 
tion between quantum launch times and time-ordered quantum arrival times. We consider 
the analytically tractable special case of exponential first passage, fold in the cost of quan- 
tum manufacture and consider capacity per unit energy (capacity per quantum). We defer 
exploration of physiologically-derived parameters applied to our results for future work. 



2 Problem Definition 



We assume that M identical quanta are emitted at times {T^}, m = 1, 2, M. The duration 
of quantum m's first-passage between source and destination is Dm- These Dm are assumed 
i.i.d. with fD„Xd) = g{d) = G'{d) where g{) is some causal probability density with mean j 
and CDF G{). We also assume that g{) contains no singularities. Thus, the first portion of 
the channel is modeled as a sum of random M- vectors 



S = T + D 



for which we have 
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and we impose an emission deadline, Tm < t, Vm G {1,2, ...,M}. The associated emission 
time ensemble probability density /T(t) is assumed causal, but otherwise arbitrary. We define 
the launch and capture of M quanta is defined as a "channel use." If we assume multiple 
independent channel uses, then the usual coding theorems apply [iS] and the channel's figure 
of merit is the mutual information between T and S, /(S;T). We will seek to understand 
the behavior of /(S; T) and provide bounds on its maximum an minimum. 

Had we imposed a mean constraint instead of a dead- 
line, the channel between T and S would be parallel ver- 



sion of the model introduced in Bits Through Queues 14 




Even so, since the quanta are identical we cannot necessar- 
ily determine which arrival corresponds to which emission 
time. Thus, the final output of the channel is a reorder- 
ing of the {sm} to obtain a set {sm} where Sm < Sm+i, 
m = 1, 2, M - 1. (See FIGURE |T1) We write this rela- 
tionship as 

S = Pn{S) (3) 

where Pk{), k = 1,2, ■ ■ ■ , Ml, is a permutation operator and 
fl is that permutation index which produces an ordered S from the argument S. 
we define Pi() as the identity permutation operator, Pi(s) = s. 

We note that the event Si = Sj {i ^ j) is of zero measure owing to the no-singularity 
assumption on g{). Thus, for analytic convenience we will assume that /s(s) = whenever 
two or more of the Sm are equal and therefore that the {sm} are strictly ordered wherever 

/sO ^ ^rn < Sm+l). 

Thus, the density fg{s) can be found by "folding" the density /s(s) about the hyperplanes 
described by one or more of the Sm equal until the resulting probability density is nonzero 



Figure 1: Quantum release chan- 
nel with reordering. 

Incidentally, 



only on the region where Sm < Sm+i, m = 1,2, M — 1. Analytically we have 

Ml 

J2fs{Pn{s)) Si<S2<---<S 
n=\ 

otherwise 



/g(s) = <J Z^^Sl^nlf;; ii^i2^---^im (^4^) 



We can likewise describe /s|x(s|t) as 

fM! 
5^/s|T(^'n(s)|t) Si<S2<---<S, 



n=l 

otherwise 



(5) 



which to emphasize the assumed causality of (?() we rewrite as 



M! 



/s|T(s|t) = i 5Zs(^n(s) -t)u(P„(s) -t) Si<S2<---<S, 

otherwise 



(6) 



where 

M 

u(P„,(s) - t) = JJ m([P„(s)]™ - t„) 

and m() is the usual unit step function. 

When (yf((i) = \e~^'^u{d), the conditional distribution on the ordered output S takes the 
particularly simple form 

M 

/g|^(s|t) = A^'^e -1 f5^u(P.(s)-t)J (7) 

for si < S2 < ■ ■ ■ < Sm. It is worth mentioning explicitly that equation ([t]) does not assume 
Si > ti as might be implicit in equation ([2]). 

With these preliminaries done, we can now begin to examine the mutual information 
between T, S and S. 



3 Mutual Information Between T and S 

The mutual information between T and S is 

/(S; T) = h{S) - h{S\T) = M {h{S) - h{S\T)) (8) 

Since the Si given the Tj are mutually independent, /i(S|T) does not depend on /T(t). 
Thus, maximization of equation ([s]) is simply a maximization of the marginal h{S) over the 
marginal frit), a problem explicitly considered and solved for a mean Tm constraint in 



The corresponding expression for the mutual information between T and S is 

J(S;T) = /i(S)-/i(S|T) (9) 

Unfortunately, /i(S|T) now does depend on the input distribution and the optimal form of 
/i(S) is non-obvious. So, rather than attempting a brute force optimization of equation (N) 



by deriving order distributions 15 , we first invoke simplifying symmetries. 

Consider that an emission vector t and any of its permutations P„(t) produce statistically 
identical outputs S owing to the reordering operation as depicted FIGUREjlj Thus, any /t() 
which optimizes equation ^ can be "balanced" to form an optimizing input distribution 
which obeys 

/T(t) = /T(Pn(t)) (10) 

for n = 1, 2, M ! and P„() the previously defined permutation operator. We will therefore 



restrict our search to "hyper-symmetric" densities /T(t) as defined by equation (10). 

If we assume /t() is hyper-symmetric, then it is easy to show that /s() must also be 
hyper-symmetric. From equation ^ we have 



/s(Pn(s)) = T"^'^ /T(t)g(P„(s) - t)dt 

Jo 

If we define t' = P~^(t) then we can write 

/s(p„(s))= r MP-\t'))gis~t')dt'= r/T(t')g(s-t')rft'=/s(s) 

Jo Jo 

The hyper-symmetry of /s(s) leads to a simple expression for /g(s). First we define Si 
as the region in s-space for which si < S2 < ■ ■ ■ < Sm- Similarly define disjoint regions Sn 
as those for which if s G 5„ then P„(s) G Si. That is, Sn is the region in s-space in which 
application of permutation operator P„() orders the components from smallest to largest. 

Following equation Q we have 

/g(s) = M!/s(s) 

for s G 5i. We can then write 

h{S) = ~[ M!/s(s)log(M!/s(s))rfs = -M! / /s(s) log /s(s)t;s - logM! 

JSi JSi 

But since /s(s) is hyper-symmetric, we also have 

h{S) = -J2 / /s(Pn(s))log/s(P„(s))rfs-logM! 

which becomes 

POO 

h{S) = - / /s(s)log/s(s)rfs-logM! = /i(S)-logM! (11) 
Jo 



We state this result theorem. 

Theorem 1 // /t() is a hyper-symmetric probability density function on emission times 
{Tm}, m = 1, 2, .., M , and the first passage density is non-singular, then the entropy of the 
size-ordered outputs S is 

h{S) = /i(S) -logM! 

Next we turn to /;.(S|T). A zero-measure edge-folding argument on the conditional density 
is not easily applicable here, so we resort to some sleight of hand. As before we define 
Q as the permutation index number that produces an ordered output from S. That is, 
Pq(S) = S G Si. Specification of the random tuple {fl, S) is equivalent to specifying S and 
vice versa. Just as in our derivation of h{S), this equivalence requires that we exclude the 
zero-measure "edges" and "corners" of the density where two or more of the Sj are equal. 

We then have, 

h{S\T) = h{Q, S|T) = h{S\T) + H{Q\S, T) (12) 
which also serves as a definition for the entropy of a joint mixed distribution {Q is discrete 



while S is continuous). We then rearrange equation (12) as 



/i(S|T) = h{S\T) - H{n\S, T) (13) 

//(i7|S, T) is the uncertainty about which Sm corresponds to which Sm given both T and S, 
and we note that 

< i/(n|S,T) < logM! (14) 

with equality on the right for any density where all the Tm are equal. 

We can then, after assuming that /t() is hyper-symmetric, write the ordered mutual 
information in an intuitively pleasing form: 



Theorem 2 



/(S; T) = /(S; T) - (log Ml - ff(fi|S, T)) (15) 



That is, an information degradation of size logM! — if(f2|S,T) < is introduced by the 
sorting operation. 

Since /;.(S|T) is a constant with respect to /T(t), maximization of mutual the information 



in equation ( 15 ) requires we maximize the expression 



/s{s) 



pcxi POO / \ 

h{S)+H{n\S,T) = -j^ /T(t)g(s-t)logn /T(v)(7(s-V)rfvjrfsdt ^^Q) 

/'OO 

+ / frr{t)H{n\S,t)dt 
Jo 

with respect to /T(t). 

Mutual information is convex in /T(t) and the space J-'t of feasible hyper-symmetric 
/T(t) is convex. That is, for any two hyper-symmetric probability functions /^^^ and f!^^ we 

^fi\t) + (1 - «:)/i'^(t) G J-T (17) 



where < n < 1. Thus, we can in principle apply variational 19 techniques to find that 
hyper-symmetric /t() which attains the unique maximum of equation However, in 
practice, direct application of this method can lead to grossly infeasible /t(), implying that 
the optimizing lies along some "edge" or in some "corner" of the convex search space. 

To proceed, we must first understand the component parts of the optimization, in partic- 
ular if(f2|S, T) and its relationship to h{S). But first the following property of expectations 
of hyper-symmetric functions over hyper-symmetric random variables will later prove useful. 
Suppose Q(x) is a hyper-symmetric function and X is a hyper-symmetric random vector. 
Then, when X is the ordered version of random vector X we have 



Q(X) =i?x[Q(X)] 



3.1 H{n\s,t) 



The optimization stated in equation (16) hinges on specification of if(f2|S,T). We first 
consider iJ(n|s, t), the admissible-permutation entropy given specific s and t. Given t, the 
probability that S produced S is 



Prob(f^|s,t) 



/s|T(s|t) 



(19) 



n=l 



where s = Pq{s). Owing to the causality of g{), some permutations will have zero probability 
since the specific s and t may render them impossible. 

Using equation ([6]), the definition of entropy and equation (19) we have 



H{Q\s,t) 



Ml 

E 

n=l 



Ml 



log 



Ml 



t) 



(20) 



and as might be imagined, equation (20) does not in general produce a closed form. 
However, for exponential g{) we can use equation ([T]) to simplify equation (19) as 



Prob(^] = A;|s,t) 



u(s- t) 



Ml 



(21) 



5^u(P„(s)-t) 



n=l 



which is a uniform probability mass function with Yln=i^iPni^) "~ t) elements. Thus, we 
can write 



M! 



i/e(^^|s,t) = logJ]u(P„(s)-t) 



(22) 



n=l 



The summation is the number of admissible permutations given s and t, and constitutes 
an upper bound for all possible causal first-passage time densities, g{). In addition, the 
exponential first passage time density is the only density which maximizes He{Q\s,t). We 
state the result as a theorem: 



Theorem 3 // we define 



then 



11=1 

H{n\s,t) < \og\n 



s,t 



with equality iff gO is exponential. 



Proof: ( Theorem |3]) Although equation (21) constitutes a proof that the exponential first 
passage time density maximizes He{^l\s, t), we can also prove the iff result directly. Consider 
that the probability mass (PMF) function of equation ( 19 ) can be written as 

Prob(^] = A:|s,t) " 



M! 

i=i 



This PMF is uniform iff 

g(P„(s)-t)=g(Pfc(s)-t) (23) 

for all n and k where Pn{s) and Pfc(s) are causal with respect to t. That is, the pairs 
(P„(s),t) and (Pfc(s),t) are admissible. Since the maximum number of non-zero probability 
Q is exactly the cardinality of admissible (P„(s),t), any density which produces a uniform 
PMF over Q thereby maximizes H{Q\s,t). 

We then note that any given permutation of a list can be achieved by sequential pairwise 
swapping of elements. Thus, equation (23) is satisfied iff 

g{xi - ti)g{x2 - t2) = g{x2 - ti)g{xi - ^2) (24) 



V admissible {(xi,a;2), (ti,t2)}- Rearranging equation (24) we have 

gjxi -ti) _ g{x2 - ti) 
g{xi-t2) g{x2-t2) 

which implies that 

gix — ti) 

— = Constant w.r.t. x 

g[x - t2) 

Differentiation with respect to x yields 

g'jx - ti) _ g{x - ti)g'{x - ta) ^ ^ 
g{x-t2) g'^{,x-t2) 



which we rearrange to obtain 
which further imphes that 
whose only solution is 



g'{x-ti) g'{x-t2] 



g[x-ti) g{x-t2) 
g'{x-h] 



g{x-ti) 



c 



g[x) (X e 



cx 



Thus, exponential g{) is the only first passage time density that can produce a maximum 
cardinality uniform distribution over Q given s and t - which completes the proof. • 

Now consider that \^\g^ is hyper-symmetric - invariant under any permutation of its 
arguments s or t. That is, 

M\ M! M! M! 

u(Pn(s) - t) = 5^ U(P„(S) - t) = ^ U(P„,(S) -t) = Y, u(Pn(s) - t) 

n=l n=l n=l n=l 

because the summation is over all M! permutations. Therefore, 

l^^ls,t=l^ls.t = l^ls,t = l^ls,t (25) 



We now enumerate admissible permutations. Owing to equation (25) we can assume 
ordered t with no loss of generality. So, let us define "bins" Bk = {t\t G [tk,tk+i)}, k = 
1, 2, M (tM+i = oo) and let bm = 1, 2, M be the bin in which Sm appears {sm G Bb,„)- 
We then define am as bin occupancies such that am = q if there are exactly q arrivals Sj G Bm- 
The benefit of this approach is that the am, do not depend on whether s or s is used. Thus, 
expectations can be taken over S whose components are mutually independent given the t. 

To calculate \^\g^ we start by defining 

m 

i=i 

Clearly rim is monotonically increasing in m with ?7o = and tim = M. We then observe 
that the am arrivals on [tm.,tm+i) can be assigned to any of the ti,t2, ■■■,tm known emission 
times except for those r]m,-i previously assigned. The number of possible new assignments is 
(m — ?7m-i)!/("^ ~ VmV- which leads to 

M , M-l 
m=l m=l 



We then define the random variable 

X 



(m) _ J 1 Si < tm+1 

otherwise 



for i = 1, 2, ...m. The PMF of X^""^ is then 



_ S G{tm+i -U) X - I 

where as previously defined, Gi) is the CDF of the causal first passage density gi) and 
G() = 1 - G() is its CCDF. We can then write 



i=l 



(m) 



and thence via equation (26), 



^s|t[|f^ls,t] = ^S|t 



M 



log(m + 1 - 



m=l 



(27) 



Since an expectation of a sum is the sum of the expectations, let us consider 



m m 



Esit [log(m + 1 - 7] J] = J2 log(m + 1 - XI n - tj)G'-''^ ( Wi - t,) (28) 

X i=l j=l 

where x is implicitly an m— ary binary vector. 

We now find it convenient to define Xj = 1 — Xj which allows us to define f]m = m — rj^ 
and thence 

M-l 



(29) 



and 



m=l 



m m 



^s|t[log(l + Vm)] = X]log(l + Y.^i)\{G^^{t^+, - t,)G^-"^(Wi - t,) (30) 
We can now define 



M-l 



m m 



j=i i=i 



(31) 



m=l X 

which can be rearranged as 



M-l M-l m 

£=1 m=£ |x|=£i=l 

We will also later find it useful to define 



(32) 



e^,,(t) = J2 llG^'it,n+l-t,)G'-'^itm+l-t, 
|x|=^i=l 



(33) 



which produces 
which after defining 
and 

can be rewritten as 



M-l 



M-1 



i/^(t) = 5^1og(l + £)5^0™,,(t) 



(34) 



e=i 



m=£ 



M-l 
m=i 



AT 



Mi — i- M 



M-l 



Tm f — r 



M/+1 



(35) 



where, once again, we have assumed that ti < t2 < ■ ■ ■ < tm- 



Finally, via equation (25 ), equation (27) and the definition of equation (31 ) in conjunction 
with Theorem [3] we have 



H{Q\S,t) < H\t) 



Theorem 4 // we define 

then since 
we have 

H{n\S,T) < H^T) 
with equality iff the first-passage time density g{) is exponential. 

3.2 HUT) 



In principle, we could derive i7''"(T) by taking the expectation of equation (34) with respect 



to ordered emission times. Although we can do just that for numerical calculations, direct 
analytic evaluation of H'^{T) requires we derive joint order densities for the T, a difficult task 
in general. Thus, for analytic simplicity we will take advantage of emission time distribution 
hypersymmetry and derive only univariate order densities. 

That is, the sum over all permutations of binary vector x in the definition of Om/(t) 
renders it hypersymmetric in ti, ...,tm given the (m + 1)^^ smallest emission time tm+i which 
for clarity we denote with the over-arrow notation. Therefore, by equation (18) we have 



E, 



(36) 



The CDF of the (m + 1)^^ smallest emission time is 

^ ]\^\ ptm+l ptm + 1 poo 



(t 



m+1 , 



1-E 



fc=0 



k 



/T(t)(itM ■ ■ ■ dtk+idtk 



k 



+ 1 -J tm + 1 
^ V ' 

M-k 



dti 
(37) 



and likewise, the CDF of the smallest unordered Ti, ...,Tm given T^+i is 

771 (. . \. \ FTi,...,Tmih, ■■■,'tm) 



Vtj < tm+1 where j = 1, m. 



-^Ti,...,T„(^m+l, ■■■,tm+l) 

Therefore, by the hypersymmetry of Qm/ in ti, ...,tm, we may write 
where 



-Pri,...,T™(^m+l, •••,^m+l 

(\ £ m 
^ 7 = 1 fc=£+l 



and thence 



M-l 



M-1 



ifnT) = ^iog(i + £)5^e^,, 



(3^ 



£=1 



And similar to the derivation of equation (35), we define 



M-l 



ml 



m=l 



and then 

to express H^{T) as 



M-l 



(39) 



4 IID T 



Our attempts at direct optimization of equation (16) have not yielded a closed form. The 



key problem is that h{S) and if(f2|T, S) are "conflicting" quantities with respect to /t()- 
That is, independence of the favors larger h{S) while tight correlation of the (as in 
Tj = Tj, i,j = 1,2, ...,M) produces the maximum if(f2|S,T) = logM!. It is this tension 
which leads to grossly infeasible ft{) (with high order singularities) when applying standard 
Lagrange-Euler variational optimization methods to equation (16). In short, a closed-form 



upper bound tighter than that provided by the data processing theorem 
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max/(S;T) < max/fS,T) 



/tO 



/tO 



(40) 



has so far eluded us. 

We therefore derive expressions for /(S;T) when the T are IID - as they must be to 
maximize /(S;T). Such an assumption has some grounding in the biology of signaling 
in that quanta (signaling molecules) are often emitted from physically distinct and separate 
repositories (vesicles). Thus, coordinating emission times could add complexity to the release 
mechanism. By deriving expressions for /(S; T) given IID T - which form lower bounds for 
maxj^/(S;T) in the case of exponential first passage times - we may provide insight for 
when the machinery necessary for tightly coordinated emissions is a worthwhile investment. 

4.1 HHT) and General IID T 



From the definition of 0^/0 in equation (33) we obtain 



m 



E: 



T<T„ 



G{T^+i - T) 



E. 



m—£ 



;i - G(T™+1 - T)) 



(41) 



for IID T. From the definition of Ff^_^^{) in equation (37) we obtain (again for IID T' 

d 



dt 



k=0 ^ ^ 



(42) 



which after rearranging as a telescoping sum simplifies to 

ff^Jt) = EZoiM-k)QfT{t)F^{t){l-FT{t)r-'-' 
- EZoik + l)^g/T(t)F^(t)(l - FTit)^-''-' 



(43) 



which further simplifies to 



M 
m + 1 



Mt)F^{t){i-FT{t)r^ 



M-m-1 



(44) 



If we then define 
we obtain 

so we can write 

Omi = (m + l) 



fT{x)G(t — x)dx 



fT{x){l - G{t - x))dx = Frit) - 4>{t) 



M 
m + l 



m 



<P\t){FT{t)-<p{t)r''dt 



and then as 



e^j = M 



M -1\ fM -£-1 
m — i 



Mm - FT{t)r'"'~YmFT{t) - mr-'dt (45) 



To evaluate equation (39) we now compute 



M-l 



Tm/ = ^ dm/ = M 



which we rewrite as 



M-l 



M-l 



FT{t)-m 

Frit) 



E(":-^") (t 



dt 



M 



hit) 



M-l-i 



FT{t)-m 



m+e 



m=0 



Frit) 

FT{t)-<t>(t) 



dt 



We consohdate the binomial sum to obtain 



M 



\-FT(t) I \l~FT(t) 



[i-FT{t)r-^[ 



M-l ( 4>{i) 

FT{t)-m 



dt 



which reduces to 



Vm/ = M 



("ill: 



hitwit) {i-m'"''~'dt 



(46) 



for 1,2,..., M-l. 

Now consider the integrand of the difference V m/ — '^ m/+i where we drop the t dependence 
for notational convenience 



M 



M-l 



1 _ s)''-'-' - M 



M-l 
£+1 



0^+1 (1 - 



.M-l~2 



We can rewrite this expression as 

M-l~l 



[M-l\ r/M-l\ /M-£-l\ /M-l\/M 



r — 1 



which after consolidating terms becomes 



M -1\ I f M 



G".)5''--( 



M - £ - 1 

r 



Extending the sum to r = and subtracting the r = term produces 
SO that 



/ r=0 



r+t 



(£ + r + l)E[0'-+^(t)] (47) 



where E[-] is the expectation using frif)- 

4.2 i/(n|S,T) Exponential First Passage 

Here we derive an expression for /(S; T) when the input distribution is that which maximizes 
/(S; T) subject to exponential first passage and an emission deadhne - we assume frif) is 
hmited to the interval [0,r]. The f^i) that maximizes h{S) was derived in 20 as 

1 



e + Ar e + Ar 

To obtain H^{T!) we calculate 



5{t) + -^—[u{t) - u{t - r)] + ^-^5{t - t) 



e + Ar 



(4J 



m = 1,2,...,M. 



1 



e + Ar 

e 

e + Ar' 




-\(t-\T) 



0<t<T 



t > T 



O.W. 



(49) 



We require an expression for the integral fT{t)(f)^{t)dt. Since j^^ 5{t)u^{t)dt = 
obtain 



we 



+ 5(t-r)(l + (e-l 



dt 



-l)u{t-T)fdt 



which reduces to 



,e + Ar, 
which further reduces to 



Ve + Ar/ k + 1 ^ \r J r + 1^ ' 



e + Ar 



fc+i 



1 ^ e 
+ Ar + 



fc+i 



A; + 1 



A; + 1 A; + 1 



and then 



1 



e + Ar 



fc+i 



k + l 



so that equation (|47|) becomes 

M 

? + 



1 E (-^; 



r=0 



M 



+ ly Ve + Ar 

' + 



M 



which reduces to 

+ Ar 

and then to 

+ Ar 

If we define k = ^ +1 and then 
and 

we can then write n 



M-i-1 
r 



1 



e + Ar 



r+e+i 



Ar + 



r + £ + 1 



A7 



M-e-i 



1 E (-1)' 

/ r=0 



e + Ar 

M-£- 1 
r 



(£ + r + 1) 



1 



e + Ar 



+ ly V e + Ar 
M 
+ 1 



1 



Ar 



e + Ar 

M-e-2 



e + Ar 



Pi 



M-e-i 



1 

e + Ar 



^+1 



M 



e + Ar 



P2 



M' 
k 



e + Ar 
1 

' e + Ar 
Ar 



1 -P2 



k 



M 



Ar + e 



A; 



p^,{l-p,f^^' (50) 



Now if we define random variables Ki to be binomial over M trials and success probability 
Pi, then we have in theorem form: 



Theorem 5 For exponential first passage with parameter X, a launch deadline constraint r 

nation 

XtM 



and the corresponding I {S]T) -maximizing launch density equation (48) we have 

Ar 



H,{n\S,T) = Ek, [\ogK^\]+EK, 



K2 



1-P2 (l-p2)(Ar + e) 



logics! 



(51) 



where Ki and K2 are a binomial random variables over M trials with success probabilities 
Pi = irk^ andp2 = respectively. 

And since the associated maximized /(S; T) is Mlog(l + ^) j20j we then have the 
following Lemma: 



Lemma 1 For exponential first passage with parameter X, a launch deadline of t and /t() 
given by equation (48) we have J(S; T) as 



M\og{l + — )-\ogM\+EK, [logKi\]+EK, 

where Ki and K2 are a binomial random variables over M trials with success probabilities 
Pi = ^ andp2 = respectively. 



K2 



X 



T 



XtM 



I-P2 (l-p2)(Ar + e) 



log 7^2! 



(52) 



5 Lower Bounds on Channel Capacity 



In the introduction we defined a channel use as the launch and capture of M quanta under 
mean and deadline constraints on emission times. We then assumed sequential (or parallel) 
independent channel uses so that the figure of merit was the mutual information /(S;T). 
Here we use the results we have derived to consider information flow limits under more 
physically plausible conditions in systems where channel uses are not so crisply defined a 
priori. 

For instance, energy is a key resource in biological systems. Thus, a good figure of merit 
for biological communication efficiency is nats/joule. In the current context, a natural defini- 
tion of capacity would be nats/quantum since signal molecule construction (often a protein 
in biological systems) requires a known amount of energy. At roughly 4 ATP per amino 
acid pi], construction of a 100-amino acid protein would require 400 ATP - a significant 
cost even in comparison to an elevated 6 x 10"^ ATP/sec total energy budget during cell 
replication (E. Coli [22]) when one considers that many signaling molecules must be pro- 
duced. Thus, it makes sense to rewrite emission time constraints as a constraint on average 
quantum production p (quanta/second). Our previous emission constraint is then 



r = r(M) 



M 
P 



(53) 



So, consider FIGURE [2] where sequential transmissions of M quanta - channel uses - are 
depicted. We will assume a "guard interval" of some duration 7(M, e) between successive 
transmissions so that all M transmissions are received before the beginning of the next 
channel use with high probability (1 — e). 



We further require that the average 
emission rate, M/ (t(M) +7(M, e)) satisfies 



1 



T(M) 



lim lim 



M 



OAf^oo r(M) +7(M,e) 



P 



(54) 



T{M) 2t(M) ktlM) 

Figure 2: Successive M-emission channel uses. 



A convenient choice of 7(M, e) is er(M) 
for any e > 0. We then require that 



lim Prob{SAf < r(M)(l + e)} 

M^oo 



(55) 



We can interpret equation (55) as given arbitrarily small e we can always find a finite M* 



such that 

Prob{sA/ < r(M)(l + e)} > 1 - e 

VM > M* . We can now derive conditions on first passage time densities under which 
equation (55) is true. 

Calculating a CDF for Sm is in general difficult since emission times Tm might be corre- 
lated. However, for a fixed emission interval [0, t(M)] we can readily calculate a worst case 
CDF for Sm and thence a deterministic upper bound on the actual signaling epoch duration 
that is satisfied with probability 1 — e. That is, for a given emission schedule t, the CDF for 
the final arrival is 

M 



771=1 

SO that 

/.t(M) M 
"^0 m=l 



However, it is easy to see that 



FgJs)>G''is-T{M))uis-TiM)) 

since G{s — tm) is monotone decreasing in tm- 
For s = r(M)(l + e) we have 

FsMM){l + e))>G'H^) (56) 

and we require 

hm G^H^) = 1 (57) 



which for convenience, we rewrite as 



Me 

lim MlogG( ) = (58) 



Thus, to satisfy equation (58), (logG'(e— )) must be asymptotically supralinear in M. 

If rewrite \ogG{^) in terms of the CCDF (5() and note that log(l — x) ~ —x for x 
small, we have 

G{ ) - e < log 1 - G{ ) < G{ ) + e 

P \ P J P 

for sufficiently large M. Thus, a first passage distribution whose CCDF satisfies 

lim MG{—) = (59) 

M^oo p 



will also allow satisfaction of equation (55) with r(M) = — and 7(M, e) = er(M). 



Since all first passage times are non-negative random variables, 

POO 

E[D] = / G{x)dx (60) 
Jo 

The integral exists iff 1/G{x) is asymptotically supralinear in x. Thus, if the mean first 



passage time E[D] exists, then equation (59) is satisfied. Finally, in the limit of vanishing e 
we have ^ 

lim lim — — r ; = lim — - — = p 

e^o Af^oo r(M) +7(M,e) ^^o 1 + e 



as required by equation (54) 



5.1 Capacity Lower Bound in Nats Per Quantum 

The maximum mutual information between T and S per quantum given M launched quanta 
with timing constraint r(M) = M/p is 

C,(M) = ^max/(S;T) (61) 

We define the limiting capacity in nats per quantum as 

Cg= lim Cg{M) (62) 

M-^oo 

Cq{M) will be monotone increasing in M since concatenation of two emission intervals with 
durations r/2 and M/2 quanta each is more constrained than a single interval of duration r 
with M quanta. 



We can derive a simple lower bound on Cg{M) by noting that equation (15) and the 



definition of equation (61) with r(M) produces 



/tO 



C<j(M) = max /(S;T) +if(fi|S,T) - logM! > max/(S; T) - logM! (63) 



/tO 



because < H{n\S,T) < M\. 

From [20] we know that the univariate maximum I{S] T) subject to T < r and a mean 
first passage time is also minimized when the mean first passage time density gi) is 
exponential with parameter A. Therefore, via Lemma [T] we have for any finite M and a finite 
launch deadline r(M), 

/(S; T) > min max /(S; T) = M log ( 1 + ) (54) 

9{) /tO V ^ J 

which means, 

C,(M) > log (1 , MM)^ _ ,85) 

for a launch deadline r(M). 

Using equation ([53]) and Stirling's approximation, logM! = MlogM — M + 0(log(M)) 



we have 

^(Mlog(l + AM)-logM!) = log^l + Am) -logM+l-^0(log(M)) 

= log (i? + ^) - i?0(log(M)) 



A 
P 

M — )■ cxD we obtain 



Defining X = the ratio of the uptake rate to the release rate, and then taking the hmit as 
;ain 

lim C,(M) = logx (66) 

A2— >c« 



We summarize the results with a theorem: 

Theorem 6 [Cq Lower Bound with Emission Deadline r(M) = ^ ] 

Given an average rate of signaling quantum production p as defined in equation (54) and 
any i.i.d. first passage time distribution with mean \^^, the timing channel capacity Cq{x) 
in nats per quantum obeys 

C,(x) >max{logx,0} (67) 

where X = p 

We emphasize that the Theorem ^ bound is general and applies to any first passage time 
density g{) with mean A~^. 

5.2 Capacity Lower Bound in Nats Per Unit Time 

The duration of a signaling epoch is t{M)+'j{M, e). Thus, for a given number M of emissions 
per channel use we define the channel capacity in nats per unit time as 

a(M) = max ——^^^-4— = ^.(M, r(M))- 



/t() r(M)+7(M,e) ' ^ ^V(M)+7(M,e) 

where the Cq{M^T{M)) explicitly denotes an emission interval of duration r(M). However, 
since we define 

M 

P - r{M)+^{M,e) ^^^^ 

we then have 

C.(M)^K:,(m,m(1-1^)) (69) 



p M 

For any given tuple (p, M, e), a positive interval duration r(M) such that all quanta are 
received by the end of the signaling epoch, r(M) + 7(M, e), either exists or does not. So, 
assume that a valid r(M) exists. We know from the previous section that 2Cq{M/2) < 
Cq{M). We also know that 

Cq{M,T{M) - a) < Cq{M,T{M)) 

for a > since increasing the emission interval cannot decrease the maximum mutual in- 
formation. We also know from the previous section that if E[D] exists, then the guard interval 

duration, 7(M, e) can be sublinear in M. So, if we set r(M) = M/p, then (^M, M (^^ - 



is an increasing function of M whose limit is Cq. We summarize with the following theorem: 



Theorem 7 If E[D] exists, then the capacity in nats per unit time of the quantum release 
timing channel obeys 

Ct = pCq (70) 



where Cq is defined in equation (62) and p is the average quantum emission rate. 



5.3 Special Case Lower Bounds: exponential first passage 



Given exponential first passage, Lemma |lj provides a lower bound on /(S; T) for a deadline 
launch constraint. We now examine 

/(S;T) 



M-5.00 M 



where we assume the launch constraint is specified by r(M) = — as in sections 
To begin, remember that 

Ar(M) = ^M = xM 



5.1 



and 



5.2 



and then note that ( J ( ) (l - 



I+XM 



M-k 



reduces to 



M{M -!)■■■ (M -k + 1) ( 1 



M 



For any finite k it is easily seen that 



Mx 



lim 



M 



M^oo \k J \l + xM 



1 



Similarly 



lim 



M 



1 



M^oo \k J \e + xM 
and we also have 



lim 



M 



M^oc \k J \e + xM 



I + XM 



e + xM 



e + xM 



M-k 



M-k 



1 /I 



e X 



e X 



k\ \X 



1 fl 



M-k 



k\ \X 



1 fe 



e X 



k\ \X 



(71) 



(72) 



(73) 



Equation (51) can be combined with equation (71), equation (72) and equation (73) to 



produce the following theorem: 

Theorem 8 [Exponential First Passage Cq lower bound: emission deadline] 

For exponential first passage and T G [0,M/p]*^, the channel capacity in nats per quan 
tum obeys 

log k\ 



1 / 1 \ 
Cg(x) >logx + e"x^ - 

k=2 



(fcx-1)- 



k\ 



(74) 



6 Discussion & Conclusion 



General lower bound (per quantum) 

General lower bound (per time) 

- — Exponential first passage (per quantum) 

■ ■ ■ ■ Exponential first passage (per time) 



We have described a basic model for a quanta timing channel wherein identical quanta are 
released and travel independently to a receiver with information conveyed by the timing of 
arrivals. We have derived general machinery for the analysis of such channels and provided 
lower bounds on channel capacity under the assumption that the mean first passage time 
between sender and receiver is finite. The lower bounds on capacity are on the order of a 
half nat per first passage time. 

It is worth noting that free diffusion (brow- 
nian motion) first passage times are not finite 
and thus not well-behaved from an informa- 
tion theoretic capacity standpoint. However, 
in an finite spatial-extent system, physical con- 
straints on quanta motion enforce finite first 
passage. It is also noteworthy that by consider- 
ing quanta in the limit of large M per signaling 
interval, our results in principle bridge the gap 
between quantum channel descriptions and sig- 
naling agent concentration-based descriptions. 
That is, though the signaling problem formu- 
lation is epoch-based (M quanta per emission 
period r, and p = M/r constant), with large 
M (and concomitantly large r), the "instanta- 
neous" concentrations of quanta within an emis- 
sion period are not so constrained. 

The question of quanta number vs. timing information is worth exploring briefly. Con- 
sider that instead of fixing the number of quanta per epoch, we might send different numbers 
of quanta in each epoch. We have shown that Cq{M) is at least linear in M. In contrast, 
the maximum amount of information conveyed per epoch by the number of quanta is exactly 
logM - strongly suhAmeax in M. The argument also applies to Ct{M) since the guard in- 
terval is proportionately larger for small M (larger-M intervals are more temporally efficient 
and therefore higher rate). Thus, in terms of information transfer, timing information seems 
strongly preferred, at least asymptotically. 




Figure 3: Lower bounds for and Ct vs. 



Our lower bounds on channel capacity in nats per quantum (equation (67) and equation 



(74)) and the corresponding bounds in nats per unit time (equation (70)) are shown FIG- 
URE |3} Increasing x increases the emission interval relative the mean first passage time 
and thereby increases the information content of any individual quantum. In addition, since 

successive quanta may be less likely to interchange position, (logM! — if(f2|S;T) 



ap- 
proaches zero. Thus, the simple lower bound of equation (67) (and correspondingly equation 
(70)) meets the lower bound for exponential first passage which has minmax /(S;T). But 



perhaps most interesting is the implication that there may exist optimum emission rates 
for a given channel as evidenced by the shape of the Ct curves in FIGURE |3] This feature 
echos 16 where an optimum burst interval for signaling molecules in a diffusive channel was 



derived. However, since we do not know the channel capacity, we do not know how tight 
our lower bounds are. It is therefore premature to say whether an optimum emission rate is 
a feature of the identical quanta timing channel. 
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